1. Field of the Invention
The present invention relates to an image processing apparatus that detects a motion vector from a plurality of images and a method thereof.
2. Description of the Related Art
Block matching for acquiring a motion vector between two images from a correlation thereof has been developed by focusing on a technique for compressing a moving image. Since such a technique for acquiring the motion vectors can acquire a shift among positions of a plurality of images that are shot in sequence, the technique can be applied to various kinds of technique fields such as electronic image stabilization processing performed by combining a plurality of images and dynamic range expansion processing.
Block matching is a method for acquiring motion vectors between the two images by moving rectangular blocks that have predetermined sizes and are set in each image to calculate the correlation of the images at a plurality of positions and by acquiring the position having the highest correlation.
FIG. 9 schematically illustrates an embodiment of block matching. Image data 901 is a reference image used for acquiring the motion vector. Image data 902 is a target image for acquiring the motion vector with respect to the image data 901.
First, an image processing apparatus sets abase block 903 that is a region as a reference in the image data 901 for acquiring the motion vector. In the image data 902, the image processing apparatus acquires a projection block 904 having a position corresponding to and the same size as that of the base block 903 in the image data 901 and, with reference to the position of the projection block 904, sets a detection area 905 that has a predetermined size and includes the projection block 904.
In the image data 902, the image processing apparatus sets a target block 906 that is included in the detection area 905 and has the same size as that of the base block 903. The image processing apparatus then obtains a correlation between the image of the base block 903 in the image data 901 and that of the target block 906 in the image data 902. The image processing apparatus further obtains a sum of absolute differences (SAD) between a signal level of image data of the base block 903 and that of the target block 906. The image processing apparatus determines that, the smaller the value of the SAD is, the higher the correlation between the images of the base block 903 and the target block 906 is.
The image processing apparatus moves the target block 906 to a whole region in the detection area 905 to acquire the SAD every time the target block 906 is moved. By detecting a position of the target block 906 having the smallest value of the SAC, in the detection area 905, the position can be acquired in which the image of the target block 906 has the highest correlation with that of the base block 903. The image processing apparatus acquires a shift between the positions of the base block 903 and the target block 906 as a motion vector of the image data of the base block 903 between the image data 901 and the image data 902. By setting a plurality of positions of the base block 903, a plurality of motion vectors between the image data 901 and the image data 902 can be acquired.
FIG. 10 illustrates an arithmetic circuit 1001 that is included in the image processing apparatus and acquires the SAD from the image data of the base block 903 and that of the target block 906.
The image processing apparatus includes the arithmetic circuit 1001, receives the image data of the base block 903 via a terminal of an input 1, and receives that of the target block 906 via a terminal of an input 2. The arithmetic circuit 1001 subtracts, from the signal level of each pixel in image data of the base block 903, the signal level of a corresponding pixel in that of the target block 906, and then adds up the absolute values to acquire the SAD.
For example, the image data of the base block 903 and that of the target block 906 each include the signals of 256 pixels of 16×16. The signal levels of each pixel of the image data of the base block 903 are defined as P(1, 1), P(1, 2), . . . , P(1, 16), P(2, 1), P(2, 2), . . . , P(16, 16). The signal levels of each pixel of the image data of the target block 906 are defined as Q(1, 1), Q(1, 2), . . . , Q(1, 16), Q(2, 1), Q(2, 2), . . . , Q(16, 16).
The arithmetic circuit 1001 can acquire the SAD from the following equation (1):
                    SAD        =                              ∑                          i              =              1                        16                    ⁢                                    ∑                              j                =                1                            16                        ⁢                                                                          P                  ⁡                                      (                                          i                      ,                      j                                        )                                                  -                                  Q                  ⁡                                      (                                          i                      ,                      j                                        )                                                                                                                        (        1        )            
If the detection area 905 is set as a region including 25×25 pixels having the projection block 904 as a center thereof, the target block 906 can move to 10 positions each vertically and horizontally. Accordingly, up to 100 SADs can be acquired in the detection area 905. Further, if the detection area 905 is set as a region including 32×32 pixels having the projection block 904 as a center thereof, the target block 906 can move to 17 positions each vertically and horizontally. Accordingly, up to 289 SADs can be acquired in the detection area 905.
As described above, the larger the ratio of the number of the pixels included in the detection area 905 to that in the base block 903 is set, the more the number of arithmetic operations for acquiring the SAD increases. Therefore, a processing time is increased. However, if the detection area 905 is decreased to decrease the number of calculations, it may not be able to detect the motion vector.